An Application of Reinforcement Learning to Aerobatic Helicopter Flight
نویسندگان
چکیده
Autonomous helicopter flight is widely regarded to be a highly challenging control problem. This paper presents the first successful autonomous completion on a real RC helicopter of the following four aerobatic maneuvers: forward flip and sideways roll at low speed, tail-in funnel, and nose-in funnel. Our experimental results significantly extend the state of the art in autonomous helicopter flight. We used the following approach: First we had a pilot fly the helicopter to help us find a helicopter dynamics model and a reward (cost) function. Then we used a reinforcement learning (optimal control) algorithm to find a controller that is optimized for the resulting model and reward function. More specifically, we used differential dynamic programming (DDP), an extension of the linear quadratic regulator (LQR).
منابع مشابه
Autonomous Inverted Helicopter Flight via Reinforcement Learning
Helicopters have highly stochastic, nonlinear, dynamics, and autonomous helicopter flight is widely regarded to be a challenging control problem. As helicopters are highly unstable at low speeds, it is particularly difficult to design controllers for low speed aerobatic maneuvers. In this paper, we describe a successful application of reinforcement learning to designing a controller for sustain...
متن کاملAutonomous Helicopter Flight via Reinforcement Learning
Autonomous helicopter flight represents a challenging control problem, with complex, noisy, dynamics. In this paper, we describe a successful application of reinforcement learning to autonomous helicopter flight. We first fit a stochastic, nonlinear model of the helicopter dynamics. We then use the model to learn to hover in place, and to fly a number of maneuvers taken from an RC helicopter co...
متن کاملReinforcement Learning with Multiple Demonstrations
Many tasks in robotics can be described as a trajectory that the robot should follow. Unfortunately, specifying the desired trajectory is often a non-trivial task. For example, when asked to describe the trajectory that a helicopter should follow to perform an aerobatic flip, one would have to not only (a) specify a complete trajectory in state space that intuitively corresponds to the aerobati...
متن کاملLearning Through Interaction
Reinforcement learning is an approach for learning optimal action policy via experiencing, i.e. using observed reward in environment states. Reinforcement learning algorithms include adaptive dynamic programming, temporal difference learning and Q-learning[1]. Examples of successful applications of reinforcement learning are controller for sustained inverted flight on an autonomous helicopter [...
متن کاملHelicopter Rotor Airloads Prediction Using CFD and Flight Test Measurement in Hover Flight
An implicit unsteady upwind solver including a mesh motion approach was applied to simulate a helicopter including body, main rotor and tail rotor in hover flight. The discretization was based on a second order finite volume approach with fluxes given by the Roeand#39;s scheme. Discretization of Geometric Conservation Laws (GCL) was devised in such a way that the three-dimensional flows on arbi...
متن کامل